In this section, we’re going to learn about what I believe is the most important part of the computer’s architecture: the memory system. Moving data from point A to point B is one of the main bottlenecks in all of today’s systems. Shown above is a picture of HBM (high-bandwidth memory) which is a relatively new memory technology primarily used in GPUs and other accelerators to try to overcome this memory bottleneck.
In this section, we’ll be talking about how memory works and memory technology. We’ll find that while there are many different memory technologies, none are “best”.
Since no one technology is the best, we want to create a memory hierarchy with caches. We will discuss the underlying principles that make caches efficient, and how hardware caches are designed.
Finally, we’ll step up a level and look at how operating systems allow multiple programs to share a single memory through virtualization.